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WHAT  SIGAR  REVIEWED 

Since  2005,  Congress  has  appropriated 
almost  $53  billion  for  the  Afghanistan 
Security  Forces  Fund— the  single  largest  U.S. 
fund  for  the  reconstruction  of  Afghanistan— to 
train,  equip,  and  sustain  the  Afghan  National 
Security  Forces  (ANSF).  The  process  of 
transferring  security  responsibility  to  the 
Afghans  is  expected  to  be  completed  by  the 
end  of  2014,  which  will  coincide  with  the 
completion  of  the  current  drawdown  of  U.S. 
and  coalition  forces.  The  North  Atlantic  Treaty 
Organization’s  Strategic  Plan  for  Afghanistan 
calls  for  the  Afghan  army  and  police  to 
achieve  a  combined  surge  strength  of 
352,000  personnel  by  the  end  of  2013,  and 
to  remain  at  that  level  through  at  least  2015, 
to  allow  for  continued  progress  toward  a 
sustainable,  secure  environment  in 
Afghanistan.  SIGAR’s  prior  work  and  the  work 
of  others  have  expressed  concern  with  the 
reliability  and  consistency  of  ANSF 
assessments.  Reliable  measures  of  the 
ANSF’s  progress  toward  becoming  a  self- 
sustaining  force  that  is  capable  of  assuming 
full  security  responsibility  for  Afghanistan  are 
critical  as  the  ANSF  assume  increasing 
responsibility  for  their  country’s  security  from 
U.S.  and  coalition  forces. 

The  objectives  of  this  audit  were  to  assess  the 
extent  to  which  (1)  the  International  Security 
Assistance  Force’s  (ISAF)  tool  for  evaluating 
the  manning,  training,  and  equipping  capacity 
of  the  ANSF— the  Commander’s  Unit 
Assessment  Tool  (CUAT)— was  consistently 
applied  to  assess  ANSF  units,  and  (2)  ISAF  is 
taking  steps  to  plan  for  the  continued 
collection,  validation,  analysis,  and  reporting 
of  ANSF  capacity  assessments  during  the 
military  drawdown  and  the  transition  of 
security  responsibility  to  the  Afghan 
government. 
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WHAT  SIGAR  FOUND 

The  National  Defense  Authorization  Act  for  Fiscal  Year  2008  directs  the 
Department  of  Defense  (DOD)  to  report  on  its  assessment  of  the  operational 
readiness  of  Afghan  National  Security  Forces  (ANSF)  units.  Specifically,  the 
Act  requires  DOD  to  report  on  the  number  of  ANSF  units  that  are  capable  of 
conducting  operations  independently,  units  that  are  capable  of  conducting 
operations  with  U.S.  and  coalition  support,  and  units  that  are  not  ready  to 
conduct  operations.  To  help  meet  this  requirement,  from  April  2010  through 
July  2013,  the  International  Security  Assistance  Force  (ISAF)  Joint  Command 
(IJC)  used  the  Commander’s  Unit  Assessment  Tool  (CUAT)  as  a  key  tool  for 
assessing  and  reporting  on  the  ANSF’s  manning,  training,  and  equipping 
capacity.  The  CUAT  was  a  two-part  assessment  of  ANSF  units.  The  first  part 
consisted  of  quantitative  data,  such  as  the  total  number  of  assigned 
personnel  and  equipment  on-hand.  IJC  advisor  teams  then  used  this  data  to 
inform  their  qualitative  evaluations,  the  second  portion  of  the  CUAT,  which 
consisted  of  one  overall  assessment  and  an  assessment  of  14  component 
assessment  areas.  Based  on  their  assessment  findings,  advisor  teams 
assigned  one  of  six  color-coded  rating  definition  levels  to  the  ANSF  units  in 
each  area  and  provided  narrative  comments  to  justify  the  assigned  rating 
definition  levels.  From  August  2011  to  July  2013,  these  rating  definition 
levels  were  (1)  independent  with  advisors,  (2)  effective  with  advisors,  (3) 
effective  with  partners,  (4)  developing  with  partners,  (5)  established,  and  (6) 
not  assessed.  CUAT  assessments,  combined  with  battlefield  reporting, 
informal  commander’s  assessments,  and  other  data,  served  as  the  basis  for 
DOD’s  and  ISAF’s  overall  assessment  of  the  ANSF’s  operational  effectiveness 
and  readiness  during  this  time. 

CUAT  rating  definition  levels  for  ANSF  units  assessed  from  January  2012  to 
July  2013  show  that  there  has  been  some  improvement  in  the  ANSF’s 
capacity  to  man,  train,  and  equip  its  forces.  For  example,  the  number  of  ANA 
units  rated  as  “independent  with  advisors”  increased  from  20  to  93  between 
January  2012  and  July  2013.  However,  the  number  of  ANA  and  ANP  units  not 
assessed  also  increased  during  this  time  due  to  the  drawdown  of  coalition 
forces. 

Between  April  2010  and  August  2011,  IJC  changed  the  CUAT  rating  definition 
level  titles  four  times.  Despite  these  changes,  SIGAR’s  analysis  of  the 
standards  supporting  each  level  shows  that  the  standards  for  the  overall 
assessment  generally  remained  consistent  from  change  to  change.  However, 
the  thresholds  included  in  the  standards  for  some  of  the  component 
assessment  areas  changed  in  some  cases.  For  example,  under  the 
September  2010  revision,  an  ANA  unit  could  achieve  a  rating  definition  level 
of  “effective  with  advisors”  for  equipment  if  its  level  of  critical  equipment  was 
greater  than  85  percent  of  its  authorized  equipment  level.  Under  the  August 
2011  revision,  an  ANA  unit  could  achieve  that  same  rating  definition  with 
“greater  than  or  equal  to”  75  percent  of  its  authorized  equipment  level. 

The  CUAT  Standard  Operating  Procedure  outlined  requirements  for  advisor 


For  more  information,  contact  SIGAR  Public  Affairs  at  (703)  545-5974  or  sigar.pentagon.ccr.mbx.public-affairs@mail.mil. 


teams  completing  the  qualitative  evaluation,  and  encouraged  advisor  teams  to  use  their  judgment  when  assessing  ANSF 
units  to  assign  rating  definition  levels  and  provide  comments  within  the  assessment  areas.  However,  the  CUAT  Standard 
Operating  Procedure  did  not  provide  clear  guidance  on  the  level  of  detail  that  is  necessary  to  support  a  team’s  rating  or 
what  the  team’s  subjective  assessments  should  contain.  This  unclear  guidance  led  to  disparities  in  the  quantity  and 
quality  of  information  across  CUAT  assessments  and  inconsistencies  in  the  evaluations  of  ANSF  units’  capacity  to  man, 
train,  and  equip  its  forces. 

In  July  2013,  IJC  replaced  the  CUAT  with  the  Regional  ANSF  Status  Report  (RASR).  IJC  officials  stated  the  command 
replaced  the  system  because  ISAF  senior  leadership  found  the  CUAT  to  be  difficult  to  read,  inconsistently  applied,  and  not 
useful.  In  addition,  IJC  officials  expect  that  the  RASR  will  simplify,  streamline,  and  improve  assessments  of  the  ANSF  by 
allowing  the  command  to  better  track  the  extent  to  which  ANSF  units’  are  able  to  employ  their  manning,  equipping,  and 
training  capacity,  which  were  the  focus  of  the  CUAT,  to  successfully  engage  the  enemy  in  combat.  The  RASR  uses  six 
color-coded  rating  definition  levels— (1)  fully  capable,  (2)  capable,  (3)  partially  capable,  (4)  developing,  (5)  established, 
and  (6)  not  assessed— to  evaluate  ANSF  units  at  the  brigade  and  regional  levels  each  month.  In  addition,  the  RASR 
focuses  on  assessing  specific  components  of  the  units,  such  as  infantry  and  communications,  against  one  overall  and  five 
targeted  categories:  combined  arms,  command  and  control,  leadership,  sustainment,  and  training.  These  five  targeted 
categories  replace  the  one  overall  assessment  and  14  component  assessment  areas  previously  used  in  the  CUAT. 

As  coalition  forces  draw  down  and  fewer  advisor  teams  are  assigned  to  ANSF  units,  IJC  officials  stated  they  will  have  less 
insight  into  the  ANSF’s  capabilities  and  will  rely  more  on  the  ANSF  for  assessment  data.  Officials  stated  this  will  decrease 
their  overall  confidence  in  ANSF  assessments  as  a  whole.  IJC,  in  coordination  with  the  Afghan  government,  is  taking  steps 
to  improve  the  ANSF’s  internal  assessment  capabilities  and  discussing  ways  to  mitigate  the  risks  of  declining  presence 
and  reduced  coalition  oversight  of  the  ANSF.  For  example,  in  July  and  August  2013,  the  Ministry  of  Interior  selected  nearly 
100  ANP  candidates  to  send  to  Turkey  to  learn  assessment  methodology  and  develop  a  new  police  assessment  tool  that 
would  standardize  assessments  across  the  ANP.  Despite  these  efforts,  ISAF  has  not  developed  a  plan  that  details  how  it 
will  (1)  ensure  the  continued  collection,  analysis,  validation,  and  reporting  of  ANSF  capability  assessments  as  coalition 
forces  drawdown  and  (2)  address  the  challenges  associated  with  having  fewer  advisor  teams  available  to  conduct 
assessments  and  relying  on  the  ANSF’s  processes  for  conducting  its  own  internal  assessments.  Without  such  a  plan,  ISAF 
may  not  be  able  to  obtain  an  accurate  understanding  of  ANSF  units’  capability— information  that  feeds  into  broader 
assessment  reports  on  the  ANSF’s  overall  effectiveness  and  readiness. 


Note:  IJC  color  coded  the  CUAT  and  RASR  rating  definition  levels  titles.  This  table  reflects 
the  command’s  color  scheme. 

Source:  IJC  response  to  SIGAR  data  call,  12/20/2012;  IJC  meeting  with  SIGAR,  4/12/13. 
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WHAT  SIGAR  RECOMMENDS 

SIGAR  is  making  one  recommendation. 
Specifically,  SIGAR  recommends  that  the  ISAF 
Commander  develop  and  implement  a  plan  for 
collecting,  validating,  analyzing,  and  reporting 
ANSF  capability  assessments  that  details  how 
ISAF  will  assess  the  ANSF’s  capability  with 
fewer  advisors  to  conduct  the  assessments 
and  mitigate  the  challenges  associated  with 
the  ANSF  reporting  on  its  own  assessment 
results. 

In  providing  comments  on  a  draft  of  this  report, 
U.S.  Force-Afghanistan  stated  that  it  “concurs 
with  the  overall”  recommendations,”  but  “non¬ 
concurs  with  some  specific  recommendations.” 
SIGAR  deleted  one  recommendation  based  on 
information  provided  in  U.S.  Forces- 
Afghanistan’s  comments.  These  comments, 
along  with  SIGAR’s  responses,  are  reproduced 
in  appendix  IV. 
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Office  of  the  Special  Inspector  General 
for  Afghanistan  Reconstruction 


February  5,  2014 

The  Honorable  Charles  T.  Hagel 
Secretary  of  Defense 

The  Honorable  Dr.  James  N.  Miller 
Under  Secretary  of  Defense  for  Policy 

General  Lloyd  J.  Austin  III 
Commander,  U.S.  Central  Command 

General  Joseph  F.  Dunford,  Jr. 

Commander,  U.S.  Forces-Afghanistan,  and 
Commander,  International  Security  Assistance  Forces 

Lieutenant  General  Mark  A.  Milley 
Commander,  International  Security  Assistance  Forces 
Joint  Command 

This  report  discusses  the  results  of  SIGAR’s  audit  to  determine  if  the  International  Security 
Assistance  Force  (ISAF)  consistently  applied  its  tools  for  evaluating  the  manning,  training,  and 
equipping  capacity  of  the  ANSF,  specifically  the  Commander’s  Unit  Assessment  Tool.  It  also 
discusses  the  steps  ISAF  is  taking  to  plan  for  the  continued  collection,  validation,  analysis,  and 
reporting  of  ANSF  capacity  assessments  during  the  military  drawdown  and  the  transition  of 
security  responsibility  to  the  Afghan  government.  It  includes  one  recommendation  to  the  ISAF 
Commander  to  develop  and  implement  a  plan  for  collecting,  validating,  analyzing,  and 
reporting  ANSF  capability  assessments  that  details  how  ISAF  will  assess  the  ANSF’s  capability 
with  fewer  advisors  to  conduct  the  assessments  and  mitigate  the  challenges  associated  with 
the  ANSF  reporting  on  its  own  assessment  results. 

U.S.  Forces-Afghanistan  (USFOR-A)  provided  comments  on  a  draft  of  this  report.  In  its 
response,  USFOR-A  stated  that  it  “concurs  with  the  overall  recommendations,”  but  “non¬ 
concurs  with  some  specific  recommendations.”  We  deleted  one  recommendation  from  our 
final  report  based  on  information  USFOR-A  provided  in  its  comments.  USFOR-A’s  comments, 
along  with  our  responses,  are  reproduced  in  appendix  IV. 

SIGAR  conducted  this  work  under  the  authority  of  Public  Law  No.  110-181,  as  amended,  and 
the  Inspector  General  Act  of  1978,  as  amended,  and  in  accordance  with  generally  accepted 
government  auditing  standards. 


John  F.  Sopko 
Special  Inspector  General 


for  Afghanistan  Reconstruction 


Mailing  2530  Crystal  Drive 
Arlington,  Virginia  22202-3940 


Tel  703  545  6000 


1550  Crystal  Drive,  9th  Floor 
Arlington,  Virginia  22202 
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Since  2005,  Congress  has  appropriated  approximately  $52.75  billion  for  the  Afghanistan  Security  Forces 
Fund— the  single  largest  U.S.  funding  resource  for  the  reconstruction  of  Afghanistan— to  train,  equip,  and 
sustain  the  Afghan  National  Security  Forces  (ANSF).1  For  fiscal  year  2013,  Congress  appropriated  $5.1  billion 
for  the  fund.  The  process  of  transferring  security  responsibility  to  the  Afghans  began  in  July  2011,  and  is 
expected  to  be  completed  by  the  end  of  2014.  Concurrent  with  this  transition  of  security  responsibility  to  the 
Afghans,  since  2011,  U.S.  and  coalition  forces  have  been  decreasing  their  military  forces  in  Afghanistan  and 
plan  to  complete  this  drawdown  process  in  December  2014. 2  Under  the  North  Atlantic  Treaty  Organization 
(NATO)3  Strategic  Plan  for  Afghanistan,  agreed  upon  at  the  Chicago  Summit  in  May  2012,  NATO  announced  its 
vision  for  a  new  mission  to  train,  advise,  and  assist  the  ANSF  after  2014,  and  a  political  commitment  to 
provide  funding  for  the  ANSF.  NATO’s  plan  calls  for  the  Afghan  army  and  police  to  achieve  a  combined  strength 
of  352,000  personnel  by  the  end  of  2013— and  to  remain  at  that  level  through  at  least  2015— to  allow  for 
continued  progress  toward  a  sustainable,  secure  environment  in  Afghanistan. 

From  2005  to  2010,  the  International  Security  Assistance  Force  (ISAF)  used  the  Capability  Milestone  (CM)  tool 
to  monitor  and  report  on  progress  in  developing  a  self-sustaining  and  capable  ANSF  by  evaluating  units  in 
areas  such  as  personnel,  equipment,  training,  and  facilities.  In  April  2010,  ISAF  implemented  the 
Commander’s  Unit  Assessment  Tool  (CUAT)  to  assess  the  manning,4  training,  and  equipping  capacity  of  the 
ANSF  on  a  quarterly  basis,  the  results  of  which  contributed  to  overall  assessments  of  the  forces’  operational 
effectiveness  and  readiness. 

Reliable  measures  of  the  ANSF’s  progress  toward  becoming  a  self-sustaining  force  that  is  capable  of  assuming 
full  security  responsibility  for  Afghanistan  are  critical  as  the  ANSF  assumes  increasing  responsibility  for  its 
country’s  security  from  U.S.  and  coalition  forces.  Further,  having  reliable  and  consistent  measures  of  the 
ANSF’s  capacity  to  man,  train,  and  equip  its  forces  is  key  to  ensuring  that  decision  makers  in  the  United  States 
and  other  coalition  countries  have  a  clear  understanding  of  the  extent  to  which  progress  is  being  made  in 
developing  the  forces.  Our  prior  work  and  the  work  of  others  have  expressed  concern  with  the  reliability  and 
consistency  of  ANSF  assessments.5 

The  objectives  of  this  audit  were  to  assess  the  extent  to  which: 

1.  ISAF’s  tool  for  evaluating  the  manning,  training,  and  equipping  capacity  of  the  ANSF— the  CUAT— was 
consistently  applied  to  assess  ANSF  units,  and 

2.  ISAF  is  taking  steps  to  plan  for  the  continued  collection,  validation,  analysis,  and  reporting  of  ANSF 
capacity  assessments  during  the  military  drawdown  and  the  transition  of  security  responsibility  to  the 
Afghan  government.6 

We  did  not  conduct  an  independent  assessment  of  the  ANSF’s  capacity  to  man,  train,  and  equip  its  forces. 
Rather,  we  determined  whether  the  tool  ISAF  used  to  assess  the  ANSF’s  capacity  to  man,  train,  and  equip  its 
forces— the  CUAT— provided  users  with  sufficient  information  to  inform  decision  making. 

To  accomplish  our  objectives,  we  reviewed  key  Department  of  Defense  (DOD)  and  ISAF  policies  and 
documents,  such  as  mandated  reports  and  relevant  standard  operating  procedures  for  conducting  ANSF 


1  The  ANSF  consists  of  the  Afghan  National  Army  (ANA)  and  Afghan  National  Police  (ANP).  Currently,  the  authorized  strength  of  the  ANSF  is 
352,000.  In  September  2013,  the  ANA  had  an  assigned  force  level  of  192,595,  including  6,778  personnel  assigned  to  the  air  force,  and 
the  ANP  had  an  assigned  force  level  of  152,336. 

2  U.S.  and  coalition  nations  plan  to  end  their  combat  role  in  Afghanistan  by  the  end  of  2014,  which  will  involve  the  withdrawal  of  a 
significant  number  of  military  forces  and  resources. 

3  The  United  States  is  one  of  28  member  countries  of  NATO. 

4  Manning  refers  to  an  inventory  of  personnel  at  an  activity  in  terms  of  numbers,  grades,  and  occupations. 

5  SIGAR  Audit  10-11,  Actions  Needed  to  Improve  the  Reliability  of  Afghan  Security  Force  Assessments,  June  29,  2010;  U.S.  Government 
Accountability  Office,  Afghanistan:  Key  Oversight  Issues,  GA0-13-218SP,  February  11,  2013;  DOD  Inspector  General,  Assessment  of  the 
U.S.  Government  Efforts  to  Train ,  Equip ,  and  Mentor  the  Expanded  Afghan  National  Police,  SP0-2011-003,  March  3,  2011. 

6  In  a  separate  review,  we  are  examining  the  methods  and  tools  used  to  determine  ANSF  personnel  strength  and  plan  to  issue  a  report  in 
the  first  half  of  2014. 
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capacity  assessments,  with  a  focus  on  the  CUAT.  We  reviewed  data  on  CUAT  results  conducted  from  April  2011 
through  July  2013,  and  analyzed  CUAT  rating  definition  levels  from  April  2010  through  July  2013.  We 
interviewed  officials  from  each  of  these  organizations  representing  several  offices,  including  ISAF  Joint 
Command’s  (IJC)  Campaign  Transition  Assessment  Group.  We  visited  select  locations  in  Kandahar,  Helmand, 
and  Kabul  provinces  in  Afghanistan  to  meet  with  coalition  advisor  teams;  observe  capacity  assessment  data 
collection,  analysis,  and  verification  processes;  and  review  capacity  assessment  documents,  which  are 
classified.  We  conducted  our  work  in  Kabul,  Helmand,  and  Kandahar  provinces  in  Afghanistan,  and 
Washington,  D.C.,  from  February  2013  to  February  2014,  in  accordance  with  generally  accepted  government 
auditing  standards.  A  more  detailed  discussion  of  our  scope  and  methodology  is  in  appendix  I. 

BACKGROUND 

The  National  Defense  Authorization  Act  for  Fiscal  Year  2008  directs  DOD  to,  among  other  things,  report  on  its 
assessment  of  the  operational  readiness  of  ANSF  units.7  Specifically,  the  Act  calls  for  DOD  to  report  on  a 
comprehensive  set  of  performance  indicators  and  measures  of  progress  toward  sustainable  long-term  security 
and  stability  in  Afghanistan,  including  the  operational  readiness  status  of  ANSF  units  that  are  (1)  capable  of 
conducting  operations  independently;  (2)  capable  of  conducting  operations  with  the  support  of  the  United 
States,  NATO  ISAF  forces,  or  other  coalition  forces;  (3)  or  not  ready  to  conduct  operations.  Further,  the  Act 
requires  DOD  to  include  in  its  report  a  description  of: 

“a  comprehensive  and  effective  long-term  strategy  and  budget,  with  defined  objectives,  for  activities 
relating  to  strengthening  the  resources,  capabilities,  and  effectiveness  of  the  Afghanistan  National 
Army  (ANA)  and  the  Afghanistan  National  Police  (ANP)  of  the  ANSF,  with  the  goal  of  ensuring  that  a 
strong  and  fully-capable  ANSF  is  able  to  independently  and  effectively  conduct  operations  and 
maintain  security  and  stability  in  Afghanistan.” 

To  comply  with  the  Act,  DOD  provides  Congress  with  its  semiannual  Report  on  Progress  Toward  Security  and 
Stability  in  Afghanistan.8 

NATO’s  transition  strategy  rests  on  supporting  an  ANSF  that  is  sustainable,  of  sufficient  size,  and  capable  of 
securing  Afghanistan.  A  capable  and  sustainable  ANSF  will  help  guarantee  that  hard-fought  security  gains  in 
the  country  will  be  preserved.  The  NATO-led  ISAF’s  main  effort  is  to  facilitate  the  conditions  under  which  the 
ANSF  can  successfully  assume  full  security  responsibility  for  Afghanistan.  As  such,  NATO  is  shifting  ISAF’s  main 
efforts  from  partnering  and  engaging  in  combat  to  advising  using  teams  that  train  and  assist  the  ANSF  at  the 
tactical  and  operational  levels  until  Afghan  forces  are  able  to  conduct  operations  independently.9  As  ANSF 
capabilities  improve,  ISAF  plans  to  adjust  its  support  levels,  allowing  the  ANSF  to  move  into  the  operational 
lead. 

From  2005  to  2010,  ISAF  used  the  CM  tool  to  monitor  and  report  on  progress  in  developing  a  self-sustaining 
and  capable  ANSF  by  evaluating  units  in  areas  such  as  personnel,  equipment,  training,  and  facilities,  with 
variations  in  assessment  areas  between  the  ANA  and  ANP.  In  June  2010,  we  reported  that  the  CM  rating 
system  created  a  disincentive  for  units  to  achieve  the  highest  rating,  CM1,  which  indicated  that  the  unit  was 
capable  of  operating  independently.10  When  an  ANSF  unit  attained  a  CM1  rating,  coalition  forces  withdrew 
assistance,  such  as  force  protection,  supplies,  and  expertise.  As  a  result,  there  was  an  incentive  for  the  unit  to 
report  that  it  was  less  capable  of  operating  independently  in  order  maintain  coalition  assistance.  We  also 


7  Pub.  L.  No.  110-181,  §1230,  122  Stat.  3,  385-90  (2008)  (amended  2013). 

8  DOD,  Report  on  Progress  Toward  Security  and  Stability  in  Afghanistan,  November  2013. 

9  According  to  IJC,  the  tactical  level  is  concerned  with  the  execution  of  battles  and  engagements,  such  as  the  Corps,  brigade,  or  kandak.  At 
the  operational  level  military  forces  attain  strategic  objectives  through  the  design,  organization,  and  conduct  of  campaigns  and  major 
operations,  such  as  National  Coordination  Centers  and  Ground  Forces  Command. 

10  SeeSIGAR  Audit-10-11. 
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noted  in  this  audit  that  the  CM  rating  system  often  overstated  unit  ratings  by  relying  on  misleading  quantitative 
measures,  such  as  the  number  of  supplies  or  assigned  personnel,  and  failing  to  take  disparities  into  account. 
For  example,  some  units  had  a  full  supply  of  vehicles  but  few  trained  drivers.  We  also  found  a  lack  of  quality 
control  as  assessment  ratings  were  reviewed  through  the  chain  of  command.  For  instance,  ANSF  capability 
ratings  regularly  used  out-dated  information,  and  high-ranking  personnel  did  not  have  access  to  the  underlying 
assessment  reports  used  to  generate  the  summary  rating  reports. 

In  April  2010,  IJC  introduced  the  CUAT  as  its  primary  tool  for  specifically  assessing  the  manning,  training,  and 
equipping  capacity  of  ANA  and  ANP  units  at  the  operational  level.11  The  CUAT  was  a  two-part  assessment  of 
ANSF  units.  The  first  portion  of  the  assessment  consisted  of  quantitative  data  that  ANSF  units  provided  directly 
to  IJC’s  Security  Force  Assistance  Advisor  Teams.  This  quantitative  data  included  the  total  number  of  personnel 
authorized,  assigned,  and  present-for-duty;  equipment  authorized,  on-hand,  and  operational;  and  required  and 
completed  training.12  IJC’s  advisor  teams  then  used  this  data  to  inform  their  qualitative  evaluations— the 
second  portion  of  the  CUAT,  which  consisted  of  one  overall  assessment  and  14  component  assessment  areas. 
The  qualitative  evaluations  were  the  advisor  teams’  subjective  assessments  of  the  unit’s  capacities  in  several 
areas,  such  as  communications,  leadership,  logistics,  operations,  and  personnel.13  For  example,  when 
assessing  a  unit’s  leadership,  advisor  teams  were  required  to  comment  on  leadership’s  competence,  unit 
morale,  and  whether  the  ANSF  unit’s  leaders  were  present  and  actively  participated  in  planning  and  execution 
of  operations.  IJC  officials  told  us  this  evaluation  also  allowed  coalition  advisors  to  describe  the  challenges, 
strengths,  weaknesses,  opportunities,  and  threats  ANSF  units  face.14 

Based  on  their  assessment  findings,  the  advisor  teams  assigned  one  of  six  color-coded  rating  definition  levels 
to  the  ANSF  units  in  each  area  and  provided  narrative  comments  to  justify  the  assigned  rating  definition  levels. 
From  August  2011  to  July  2013,  these  rating  definition  levels  were  (1)  independent  with  advisors  (green),  (2) 
effective  with  advisors  (yellow),  (3)  effective  with  partners  (orange),  (4)  developing  with  partners  (red),  (5) 
established  (gray),  and  (6)  not  assessed  (no  color).15 

IJC  scheduled  CUAT  assessments  on  a  quarterly  basis.  The  CUAT  assessments  helped  inform  ISAF  and  the 
Regional  Commands  of  current  issues  regarding  the  development  and  sustainment  of  the  ANSF.  Military 
officials  also  used  these  assessments  to  determine  where  to  place  coalition  advisor  teams.  In  addition,  IJC 
used  the  results  of  CUAT  assessments  along  with  other  information,  such  as  battlefield  reporting,  informal 
commander’s  assessments,  and  ISAF’s  assessments  of  the  Ministries  of  Defense  and  Interior— the  ministries 
that  oversee  the  ANA  and  ANP,  respectively— to  develop  an  overall  assessment  of  the  ANSF’s  operational 
effectiveness  and  readiness.  This  overall  assessment  was  intended  to  provide  stakeholders  with  a  regular 
update  on  the  ANSF’s  overall  progress  toward  becoming  a  self-sustaining  force  that  is  capable  of  assuming  full 
security  responsibility  for  Afghanistan  at  the  end  of  2014.  NATO  and  ISAF  compiled  and  reported  the  results  of 
these  assessments  to  a  wide  range  of  stakeholders,  such  as  the  White  Flouse,  DOD,  Congress  and  NATO,  to 
inform  decision  making.  Figure  1  demonstrates  the  process  for  collecting  and  reporting  ANSF  assessments. 


11  IJC  officials  indicated  that  the  strategic  level  is  concerned  with  national  or,  in  specific  cases,  alliance  or  coalition  objectives,  and  includes 
the  Ministry  of  Defense,  Ministry  of  Interior,  and  the  ANA  General  Staff. 

12  This  audit  focused  on  the  qualitative  portion  of  the  CUAT.  However,  during  the  course  of  the  audit,  we  identified  several  weaknesses 
affecting  the  reliability  of  the  quantitative  data  the  ANSF  provides  ISAF,  such  as  issues  associated  with  manual  reporting  and  low  levels  of 
literacy  and  numeracy  within  the  ANSF.  We  plan  to  issue  a  subsequent  audit  report  on  the  reliability  of  ANSF-provided  data,  with  a  focus  on 
personnel  data. 

13  See  appendix  II  for  a  list  of  the  assessment  areas  and  topics  included  in  the  qualitative  portion  of  the  CUAT. 

14  Individual  CUAT  assessments  and  ratings  are  classified. 

15  See  appendix  III  for  a  detailed  description  of  the  overall  CUAT  rating  definition  level  standards  implemented  in  August  2011. 
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Figure  1  -  Flow  of  ANSF  Assessment  Data 


CUAT  Data  Flow 
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Note:  CUAT  data  was  typically  compiled  with  other  information,  such  as  battlefield  reporting  and 
commander’s  assessments,  to  develop  an  overall  assessment  of  the  ANSF. 

Source:  SIGAR  analysis  of  IJC  documents. 


ALTHOUGH  CUAT  RESULTS  SHOW  THAT  ANSF  CAPACITY  HAS  IMPROVED, 
UNCLEAR  GUIDANCE  RESULTED  IN  INCONSISTENCIES  IN  THE  QUALITY  OF  DATA 
SUPPORTING  ASSIGNED  RATING  DEFINITION  LEVELS 

CUAT  Assessment  Results  Have  Shown  that  ANSF  Capacity  Has  Improved 

CUAT  rating  definition  levels  for  ANSF  units  assessed  from  January  2012  to  July  2013  show  that  there  has 
been  some  improvement  in  the  ANSF’s  capacity  to  man,  train,  and  equip  its  forces.  Specifically,  the  number  of 
ANA  units  rated  as  “independent  with  advisors”  increased  from  20  to  93  between  January  2012  and  July 
2013.  The  number  of  units  “not  assessed,”  however,  also  increased  from  4  to  121  during  that  time  as 
coalition  forces  have  drawn  down.  The  number  of  ANP  units  rated  as  “independent  with  advisors”  has 
decreased  by  1  between  January  2012  and  July  2013,  and,  similar  to  the  ANA,  the  number  of  units  “not 
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assessed"  also  increased  from  24  to  401.  Tables  1  and  2  show  a  breakdown  of  the  CUAT  ratings  for  ANA  and 
ANP  units  assessed  from  January  2012  to  July  2012. 


Table  1  -  CUAT  Rating  Definition  Levels  for  ANA  Units  Assessed  from  January  2012  to  July  2013 


Rating  Definition  Level 

Cycle 

Independent 

with  Advisors 

Effective  with 

Advisors 

Effective  with 

Partners 

Developing  with 

Partners 

Established 

Not  Assessed 

Cycle  14 
(Jan-Apr  2012) 

20 

118 

49 

10 

3 

4 

Cycle  15 
(Apr-Jul  2012) 

30 

135 

39 

11 

1 

51 

Cycle  16 
(Jul-Oct  2012) 

41 

127 

35 

7 

1 

81 

Cycle  17 

(Oct  2012-Jan  2013) 

40 

134 

19 

7 

2 

100 

Cycle  18 
(Jan-Apr  2013) 

67 

108 

17 

4 

1 

115 

Cycle  19 
(Apr-Jul  2013) 

93 

89 

8 

2 

3 

121 

Note:  This  data  reflects  the  CUAT  rating  definition  levels  implemented  in  August  2011. 


Source:  SIGAR  analysis  of  IJC  data. 


Table  2  -  CUAT  Rating  Definition  Levels  for  ANP  Units  Assessed  from  January  2012  to  July  2013 


Rating  Definition  Level 

Cycle 

Independent  with 

Advisors 

Effective 

with  Advisors 

Effective 

with  Partners 

Developing 

with  Partners 

Established 

Not  Assessed 

Cycle  14 
(Jan-Apr  2012) 

65 

168 

81 

23 

5 

24 

Cycle  15 
(Apr-Jul  2012) 

49 

130 

61 

31 

6 

131 

Cycle  16 
(Jul-Oct  2012) 

71 

101 

45 

15 

3 

301 

Cycle  17 

(Oct  2012-Jan  2013) 

43 

86 

26 

3 

2 

347 

Cycle  18 
(Jan-Apr  2013) 

57 

80 

14 

0 

1 

363 

Cycle  19 
(Apr-Jul  2013) 

64 

43 

8 

1 

0 

401 

Note:  This  data  reflects  the  CUAT  rating  definition  levels  implemented  in  August  2011. 

Source:  SIGAR  analysis  of  IJC  data. 
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CUAT  Rating  Definition  Levels  Changed  Four  Times,  but  the  Standards  for  Each 
Level  Generally  Stayed  Consistent 

Since  the  CUAT’s  inception  in  2010,  IJC  has  changed  the  CUAT  rating  definition  levels  four  times.  According  to 
IJC  officials,  the  changes  were  necessary  because,  overtime,  rating  definition  levels  had  become  artificially 
inflated,  and  officials  conducting  the  assessments  had  a  limited  understanding  of  the  differences  between 
rating  definition  levels.  Officials  stated  the  new  levels  would  better  enable  IJC  to  determine  when  a  particular 
ANSF  unit  no  longer  needs  coalition  assistance,  allowing  the  command  to  reassign  advisor  teams  to  less 
developed  or  newly  formed  units.  Table  3  shows  the  changes  to  the  CUAT  rating  definition  level  titles  from  April 
2010  to  August  2011. 

Table  3  -  History  of  Changes  to  CUAT  Rating  Definition  Level  Titles,  April  2010  to 
August  20 11 


CUAT 


April  2010 

July  2010 

September  2010 

October  2010 

August  2011 

Effective  with 

Advisors 

Independent 

Independent 

Independent 

Independent 
with  Advisors 

Effective  with 

Assistance 

Effective  with 

Advisors 

Effective  with 
Advisors 

Effective  with 

Advisors 

Effective  with 

Advisors 

Dependent  on 
Coalition  Forces 
for  Success 

Effective  with 

Assistance 

Effective  with 

Assistance 

Effective  with 

Assistance 

Effective  with 

Partners 

Barely  Effective 

Dependent  on 
Coalition  Forces 
for  Success 

Dependent  on 
Coalition  Forces 
for  Success 

Developing 

Developing  with 
Partners 

Ineffective 

Ineffective 

Ineffective 

Established 

Established 

Not  Assessed 

Not  Assessed 

Not  Assessed 

Not  Assessed 

Not  Assessed 

Note:  IJC  color  coded  the  CUAT  rating  definition  level  titles.  This  table  reflects  the  command’s  color 
scheme  as  it  changed  those  titles. 

Source:  IJC  response  to  SIGAR  data  call,  12/20/2012;  IJC  meeting  with  SIGAR,  4/12/13. 

Our  analysis  of  the  CUAT  rating  definition  levels  showed  that  although  IJC  used  four  different  rating  definition 
level  titles— the  September  2010  change  maintained  the  July  2010  rating  definition  levels,  but  changed  the 
color  coding— the  standards  supporting  each  level  for  the  overall  assessment  generally  remained  consistent 
from  change  to  change.  However,  the  thresholds  included  in  the  standards  for  the  14  component  assessment 
areas  changed  in  some  cases.  For  example,  under  the  September  2010  revision,  an  ANA  unit  could  achieve  a 
rating  definition  level  of  “effective  with  advisors”  for  equipment  if  its  level  of  critical  equipment  was  greater 
than  85  percent  of  its  authorized  equipment  level.  Under  the  August  2011  revision,  an  ANA  unit  could  achieve 
that  same  rating  definition  with  “greater  than  or  equal  to”  75  percent  of  its  authorized  equipment  level, 
representing  a  decrease  in  the  amount  of  equipment  required.  In  other  cases,  the  thresholds  increased.  For 
example  under  the  April  2010  rating  definition  levels,  an  ANP  unit  could  achieve  a  rating  definition  level  of 
“dependent  on  coalition  forces  for  success”— the  third  highest  rating  definition  level— for  training  if  greater  than 
33  percent  of  personnel  had  undergone  basic  training.  Under  the  subsequent  revision  in  July  2010,  this  unit 
would  have  had  to  have  at  least  50  percent  of  personnel  complete  basic  training  to  maintain  a  third  tier  rating 
definition  level,  which  had  changed  to  “effective  with  assistance.” 


SIGAR  14-33-AR/ANSF  Capacity  Assessments 


Page  6 


IJC  Did  Not  Provide  Advisor  Teams  with  Guidance  on  the  Level  of  Detail  Required  To 
Support  ANSF  Unit  Rating  Definition  Levels  Assigned  Using  the  CUAT 


As  previously  noted,  the  CUAT  was  a  two-part  assessment  of  ANSF  units  that  consisted  of  quantitative  data  that 
ANSF  units  provided  directly  to  IJC’s  advisor  teams  and  the  advisor  teams’  qualitative  evaluations  of  the  units 
on  one  overall  assessment  and  14  component  assessment  areas.  The  CUAT  Standard  Operating  Procedure 
outlined  requirements  for  advisor  teams  completing  the  qualitative  evaluation,  and  encouraged  advisor  teams 
to  use  their  military  judgment  when  assessing  ANSF  units  to  assign  rating  definition  levels  and  provide 
comments  within  the  assessment  areas.  Flowever,  the  CUAT  Standard  Operating  Procedure  did  not  provide 
specific  criteria  defining  the  level  of  detail  that  was  necessary  to  support  a  unit’s  assigned  rating  definition 
level  or  what  these  subjective  portions  of  the  assessment  should  contain.  In  addition,  the  fragmentary 
orders16— issued  at  the  beginning  of  each  CUAT  cycle  to  provide  advisor  teams  with  timelines  for  conducting 
the  assessments  and  submitting  their  reports— did  not  clearly  describe  the  level  of  detail  for  advisor  teams  to 
include  in  the  comments  supporting  their  assigned  rating  definition  levels.17  Further,  some  regional  commands 
interpreted  the  CUAT  Standard  Operating  Procedure  for  completing  the  qualitative  portion  of  the  CUAT 
differently.  This  lack  of  clear  guidance  led  to  inconsistencies  in  the  narrative  comments  included  in  the 
qualitative  evaluations  of  ANSF  units.  For  example,  IJC  officials  told  us  that  advisors  in  Regional  Command- 
North  commonly  provided  succinct  responses,  such  as  a  “yes"  or  “no,"  to  the  qualitative  topics  without 
additional  details.  In  contrast,  advisors  in  Regional  Command-West  provided  detailed  responses  that 
sometimes  did  not  address  the  individual  topics.  Further,  IJC  officials  stated  that  some  advisor  teams’ 
comments  only  partially  responded  to  the  assessment  topics. 

IJC  officials  also  stated  there  was  a  tendency  for  new  advisor  teams  to  rate  ANSF  units  lower  than  their 
immediate  predecessors.  Officials  told  us  this  was  because  new  teams  approached  assessments  without 
access  to  the  observations  the  prior  teams  had  made  over  time,  resulting  in  varying  rating  definition  levels.  To 
address  this  issue,  some  regional  commanders  took  it  upon  themselves  to  review  assessments  before 
approving  the  results,  in  order  to  avoid  decreases  in  unit  rating  definition  levels  that  were  not  supported  with 
appropriate  justification.  As  an  additional  control,  officials  told  us  IJC  reviewed  all  unit  assessments  that 
resulted  in  a  change  in  rating  definition  level.  As  part  of  this  review,  the  command  examined  the  justification  in 
the  narratives  to  ensure  it  supported  the  rating  change.  If  the  justification  did  not  support  the  rating  change  or 
was  not  clear  to  the  reviewer,  IJC  solicited  additional  information  from  the  responsible  advisor  team.  IJC 
officials  stated  that  in  cases  in  which  a  unit’s  ratings  were  not  changed,  they  did  not  review  the  assessments 
due  to  limited  manpower  and  the  volume  of  assessments  submitted  each  cycle. 


IJC  Replaced  the  CUAT  with  a  New  Assessment  System  in  July  2013 

The  IJC  replaced  the  CUAT  with  a  new  monthly  assessment  system,  known  as  the  Regional  ANSF  Status  Report 
(RASR),  in  July  2013. 18  According  to  IJC’s  Commanding  General,  the  CUAT  was  replaced  because  it  was 
inconsistent  and  not  useful  to  leadership.  The  RASR  is  intended  to  improve  upon  the  CUAT  by  allowing  IJC  to, 
for  example,  (1)  better  track  the  extent  to  which  ANSF  units  are  able  to  employ  their  capacity— manning, 
equipping,  and  training,  which  were  the  focus  of  the  CUAT— to  successfully  engage  the  enemy  in  combat  in  an 
effort  to  measure  the  capability  of  those  units;  (2)  adapt  advisor  teams  to  resource  constraints;  and  (3)  better 
assess  the  varying  branches  of  the  ANSF.  The  RASR  uses  six  color-coded  rating  definition  levels— (1) 
independent  with  advisors  (green),  (2)  effective  with  advisors  (yellow),  (3)  effective  with  partners  (orange),  (4) 
developing  with  partners  (red),  (5)  established  (gray),  and  (6)  not  assessed  (no  color)— to  assess  ANSF  unit 


16  A  fragmentary  order  is  a  military  order  used  to  send  timely  changes  of  existing  orders  to  subordinate  and  supporting  commanders  while 
providing  notification  to  higher  and  other  applicable  commands. 

17  The  fragmentary  orders  issued  for  each  CUAT  reporting  cycle  are  classified.  Therefore,  we  could  not  report  on  the  specific  contents  of  the 
orders. 

18  With  the  exception  of  what  we  are  reporting,  much  of  the  information  on  the  RASR  is  currently  classified.  Therefore,  we  could  not  report 
specific  data  and  guidance  at  the  time  we  completed  our  review. 
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capabilities  at  the  brigade  and  regional  levels  on  a  monthly  basis.19  In  addition,  the  RASR  will  focus  on 
assessing  specific  ANA  and  ANP  components,  such  as  infantry  and  communications,  against  one  overall  and 
five  targeted  categories— combined  arms,  command  and  control,  leadership,  sustainment,  and  training- 
depending  on  the  individual  component,  rather  than  the  one  overall  assessment  and  14  component 
assessment  areas  in  the  CUAT’s  qualitative  evaluation.  IJC  officials  stated  the  RASR  is  more  focused  than  the 
CUAT  because  it  targets  fewer  units  with  greater  consistency  by  assessing  the  same  units  each  month.  This 
approach,  IJC  officials  stated,  allows  the  command  to  better  identify  trends  in  unit  capabilities,  in  contrast  with 
the  CUAT,  with  which  advisor  teams  assessed  units  on  a  quarterly  cycle  but  did  not  necessarily  assess  every 
unit  each  quarter.  In  addition,  according  to  U.S.  Forces-Afghanistan’s  (USFOR-A)  comments  on  a  draft  version 
of  this  report,  IJC  issued  a  fragmentary  order  on  September  30,  2013,  with  specific  instructions  on  the 
information  advisor  units  are  to  include  in  the  comments  supporting  their  assigned  rating  in  each  category  for 
the  assessed  unit.  USFOR-A’s  comments  also  noted  that  IJC  conducts  a  quality  control  check  of  every  narrative 
to  ensure  adequate  information  is  provided. 

DOD  LACKS  A  PLAN  FOR  COLLECTING,  VALIDATING,  ANALYZING,  AND 
REPORTING  ON  ANSF  ASSESSMENTS  DURING  THE  DRAWDOWN 

As  noted,  IJC  has  shifted  its  ANSF  unit-level  assessment  from  focusing  on  capacity,  as  was  done  under  the 
CUAT,  to  focusing  on  units’  capabilities  under  the  RASR.  Although  ISAF  is  taking  some  steps  to  mitigate  the 
risks  associated  with  decreased  insight  into  the  ANSF’s  capability  as  the  military  drawdown  continues,  ISAF  has 
not  developed  a  formal  plan  that  details  how  it  will  ensure  the  continued  collection,  validation,  analysis,  and 
reporting  of  accurate,  reliable,  and  useful  ANSF  assessment  data  during  this  time.  IJC  plans  to  assign  fewer 
advisor  teams  to  the  kandak-level  of  the  ANA,  focusing  instead  on  advising  at  the  higher  brigade  and  corps 
levels.20  Similarly,  IJC  will  be  assigning  fewer  advisor  teams  at  the  ANP  provincial  headquarters,  instead 
planning  to  focus  its  advising  efforts  on  the  regional  headquarters  level.21  Thus,  the  ANSF  will  have  a  steadily 
decreasing  number  of  advised  units,  which  will  result  in  a  reduction  in  the  number  of  assessments  conducted. 

IJC  experienced  the  negative  impact  of  the  decrease  in  the  number  of  IJC  advisor  teams  on  the  number  of 
ANSF  units  assessed  when  it  still  used  the  CUAT.  From  January  to  April  2012,  IJC  advisor  teams  assessed  98 
percent  of  ANA  units.  During  that  same  time  period  the  following  year,  advisor  teams  only  assessed  63  percent 
of  ANA  units.  Table  4  shows  the  declining  number  of  CUAT  assessments  completed  from  January  2012  through 
July  2013. 


19  When  IJC  developed  these  rating  definition  levels  in  July  2013,  the  command  initially  intended  to  use  them  under  the  CUAT.  This  would 
have  constituted  a  fifth  change  in  the  CUAT  rating  definition  levels  since  April  2010.  However,  the  command  did  not  implement  this  change 
under  the  CUAT,  instead  opting  to  use  these  rating  definition  levels  when  it  introduced  the  RASR. 

20  Combat  forces  form  the  basic  operational  arm  of  the  ANA  and  are  divided  into  six  corps  and  one  division,  each  responsible  for  a  specific 
part  of  Afghanistan.  Each  ANA  corps  or  division  is  made  up  of  one  to  four  brigades,  and  each  brigade  typically  consists  of  six  kandaks.  A 
kandak  consists  of  approximately  800  soldiers  and  is  the  Afghan  equivalent  to  a  U.S.  Army  battalion. 

21  According  to  IJC,  the  command  will  advise  the  Afghan  Uniformed  Police  at  the  “Type  A”  headquarters,  which  are  regional  headquarters 
that  recently  replaced  the  zone  headquarters  structure  but  reside  in  the  same  six  provinces  as  the  zone  headquarters. 
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Table  4  -  Number  of  CUAT  Assessments  Completed  during  Cycle  14  through  Cycle  19 
(January  2012  to  July  2013) 

ANA 

ANP 

Cycle  Dates 

Assessed  Units 

%  of  Total  ANA  Units 

Assessed  Units 

%  of  Total  ANP  Units 

Cycle  14 

1/2012-4/2012 

200 

98% 

342 

93% 

Cycle  15 

4/2012-7/2012 

216 

81% 

277 

68% 

Cycle  16 

7/2012-10/2012 

211 

72% 

235 

44% 

Cycle  17 

10/2012-1/2013 

202 

67% 

160 

32% 

Cycle  18 

1/2013-4/2013 

197 

63% 

152 

30% 

Cycle  19a 

4/2013-7/2013 

195 

62% 

116 

22% 

Note:  CUAT  assessments  were  conducted  on  a  quarterly  basis. 

a  Estimated.  IJC  expects  the  total  number  of  ANA  and  ANP  units  in  Cycle  19  to  remain  the  same  as  Cycle  18  and  up  to  a  10 

percent  decrease  in  the  number  of  assessed  units,  due  to  a  reduction  in  the  number  of  advisor  teams  during  Cycle  19. 

Source:  SIGAR  analysis  of  ISAF  data. 

Further,  IJC  officials  told  us  that  as  ISAF  continues  to  draw  down  and  advises  a  decreasing  number  of  ANSF 
units,  their  “overall  confidence  in  the  overall  assessment  of  the  ANSF  as  a  whole  decreases.”  According  to 
these  officials,  as  assessment  information  collected  loses  fidelity,  ISAF’s  ability  to  maintain  its  current  level  of 
insight  into  the  ANSF’s  capabilities  will  decrease,  resulting  in  the  need  to  rely  more  heavily  on  the  Afghans  for 
both  the  quantitative  and  qualitative  portions  of  the  assessments. 

ISAF  is  currently  working  with  the  ANSF  to  ensure  that  timely,  accurate,  and  mutually  beneficial  information  can 
be  collected  and  transmitted  throughout  the  ANSF  command  structure.  This  effort  is  intended  to  facilitate  the 
ANSF’s  ability  to  effectively  track  and  manage  its  own  training  and  readiness  status  and  to  assess  itself  in  an 
effort  to  ensure  that  it  can  sustain  these  assessments  beyond  2014.  For  example,  IJC’s  ANSF  Internal 
Assessment  Mechanism  information  papers— which  provide  updates  on  efforts  to  improve  the  quality  and 
reliability  of  assessments  as  reporting  shifts  to  the  ANSF— report  that  as  coalition  forces  decline  and  insight 
into  ANSF  units  reduces,  IJC  is  working  to  enhance  ANSF  internal  assessments  by  October  2014.  Enhanced 
internal  assessments  will  enable  IJC  to  shift  from  advisor  team  reporting  to  ANSF  reporting.  Internal 
assessments  are  not  new  to  the  ANSF,  as  the  ANA  implemented  its  Readiness  Reporting  System  in  April  2005, 
and  the  ANP  implemented  its  Force  Readiness  Report  in  December  2010.  However,  IJC  reports  that  these 
ANSF  readiness  systems  are  complicated,  out  of  date,  poor  at  assessing  or  measuring  effectiveness,  and 
lacking  in  true  measures  of  literacy,  leader  qualifications,  operational  and  sustainment  systems  effectiveness, 
and  long-term  sustainability.  IJC,  the  NATO  Training  Mission-Afghanistan,  and  the  International  Police 
Coordination  Board  are  coordinating  with  the  ANSF  to  refine  the  systems  to  make  them  more  efficient,  user- 
friendly,  and  compatible  with  the  reporting  needs  of  the  international  community.22 

IJC  also  reports  that  operational  planning  teams  meet  regularly  to  work  to  improve  effectiveness  of  these 
systems.  The  teams’  goal  is  for  the  coalition  to  be  confident  in  the  adequacy  of  the  ANSF  reports  produced 
under  these  systems  by  October  2014.  In  addition,  IJC’s  ANSF  Internal  Assessment  Mechanism  paper 
indicates  that  the  command  plans  to  conduct  three  validation  checks  of  the  reliability  of  the  Afghan  internal 
readiness  reports  by  comparing  the  results  ANA’s  Readiness  Reporting  System  and  the  ANP’s  Force  Readiness 
Report  against  the  results  of  the  RASR.  IJC  conducted  the  first  check  in  August  2013,  and  plans  to  conduct  the 
second  check  in  February  2014,  and  the  third  check  in  August  2014.  According  to  IJC  officials,  the  command 
received  the  ANSF’s  internal  assessments  data  for  the  August  2013  validation  check  in  October  2013,  and 
plans  to  complete  its  analysis  of  this  data  by  the  end  of  December  2013. 

According  to  officials,  because  IJC  will  increasingly  rely  on  the  ANSF’s  reporting,  the  command  is  working  with 
the  ANA  and  ANP  to  develop  the  Afghan  government’s  ability  to  conduct  internal  assessments  at  the 


22  The  NATO  Training  Mission-Afghanistan  is  tasked  with  overseeing  training  and  equipping  of  Afghan  forces  throughout  Afghanistan.  The 
International  Police  Coordination  Board  acts  as  the  main  coordination  board  for  institutional  and  police  reform  within  Afghanistan. 
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operational  level.  For  example,  IJC  officials  told  us  that  the  ANA  created  168  tashkil23  positions  for  literate 
officers  to  conduct  assessment  reporting  for  their  own  units.  In  addition,  in  July  and  August  2013,  the  Ministry 
of  Interior  selected  nearly  100  ANP  candidates  to  send  to  Turkey  to  learn  assessment  methodology  and 
develop  a  new  police  assessment  tool  with  the  goal  of  standardizing  assessments  across  the  ANP.24 

Despite  these  efforts,  ISAF  does  not  have  a  formal  plan  that  details  how  it  will  ensure  the  continued  collection, 
validation,  analysis,  and  reporting  of  accurate,  reliable,  and  useful  ANSF  capability  assessment  data  as 
coalition  forces  drawdown.  U.S.  Central  Command  and  Joint  Staff  officials  expressed  concern  over  the  ability  of 
coalition  forces  to  continue  to  collect  useful  ANSF  data  during  and  after  the  drawdown.  U.S.  Central  Command 
officials  told  us  they  meet  with  ISAF  and  NATO  counterparts  biweekly  to  discuss  and  mitigate  pending 
assessment  challenges,  such  as  resourcing,  accountability,  and  reliability  of  data  that  will  arise  from  reduced 
coalition  force  levels  in  Afghanistan.  They  also  informed  us  that  a  task  list  designed  to  mitigate  some  of  these 
challenges  was  developed  following  a  November  2012  Afghanistan  Assessments  Conference.  The  task  list 
identifies  nine  discreet  tasks,  including  developing  a  plan  for  collecting  data  for  ANSF  assessments  and 
continuing  to  develop  the  Ministry  of  Defense’s  and  the  Ministry  of  Interior’s  assessment  abilities.  In  addition, 
IJC  officials  stated  that  ISAF  is  considering  alternatives  for  the  continued  assessment  of  ANSF  units,  such  as 
using  advisor  teams  to  conduct  spot  assessments  of  particular  ANSF  units.  The  use  of  spot  assessments  had 
not  been  confirmed  or  documented  at  the  time  we  completed  our  review. 

The  National  Defense  Authorization  Act  for  Fiscal  Year  2008,  as  amended,  requires  DOD  to  report  on  a  long¬ 
term,  detailed  plan  for  sustaining  the  ANSF.  This  plan  is  to  include  defined  objectives  and  any  actions  taken  to 
assist  the  Afghan  government  in  establishing  strong  ANSF  readiness  assessment  tools  and  metrics,  and  the 
results  of  these  actions.25  DOD’s  2010  sustainment  plan  included  actions  taken  to  continue  assessment  of 
the  ANSF  based  on  recommendations  from  the  Commander  of  ISAF.26  However,  because  a  drawdown  had  not 
been  announced  at  the  time  of  the  2010  plan  issuance,  that  sustainment  plan  did  not  consider  the  possible 
impact  on  the  ability  of  coalition  forces  to  collect,  analyze,  verify,  or  report  ANSF  assessments  with  fewer 
available  resources.  DOD  and  ISAF  officials  told  us  they  have  not  developed  a  successor  to  the  2010 
sustainment  plan  that  addresses  the  collection,  analysis,  verification  and  reporting  of  ANSF  assessments 
during  the  drawdown.27 

Because  ISAF  will  have  fewer  available  coalition  teams  and  be  relying  increasingly  on  ANSF-provided 
information,  it  is  important  for  ISAF  to  have  a  plan  detailing  how  it  will  assess  the  ANSF’s  capability  and 
address  the  challenges  associated  with  the  ANSF  reporting  its  own  assessment  results.  Without  a  plan  for 
collecting,  validating,  analyzing,  and  reporting  on  ANSF  assessments  during  the  military  drawdown,  ISAF  may 
not  be  able  to  obtain  an  accurate  understanding  of  ANSF  units’  capability— information  that  feeds  into  broader 
assessment  reports  on  the  ANSF’s  overall  effectiveness  and  readiness.28 

CONCLUSION 

As  IJC  increasingly  focuses  its  efforts  on  assessing  the  capability  of  ANSF  units  at  the  brigade  and  regional 
levels  using  its  new  RASR  assessment  system  in  place  of  the  CUAT,  it  is  imperative  that  IJC  develop  plans  that 
were  lacking  under  the  CUAT  in  order  to  better  ensure  that  the  RASR  will  provide  consistent  and  reliable 


23  The  tashkil  is  the  Afghan  government’s  official  list  of  required  ANA  and  ANP  personnel  by  position  and  rank. 

24  Candidates  for  the  ANP  assessment  training  course  in  Turkey  will  include  30  police  officers  from  the  Ministry  of  Interior  as  well  as  an 
additional  68  police  officers— 2  from  each  province. 

25  Pub.  L.  No.  110-181,  §1231,  122  Stat.  3,  390-91  (2008)  (amended  2013). 

26  ISAF  ultimately  tasked  IJC  with  undertaking  these  assessments. 

27  Although  DOD’s  reports  to  Congress  in  April  2012  and  November  2013  include  reporting  on  ANSF  assessment  efforts,  DOD  and  ISAF 
officials  did  not  identify  either  of  these  as  a  successor  to  the  April  2010  sustainment  plan. 

28  DOD  officials  told  us  that  they  are  awaiting  strategic  direction  from  the  President  regarding  the  post-2014  Security  Force  Assistance 
mission’s  strength  and  structure  before  planning  can  begin. 
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reporting  of  the  ANSF’s  capability.  As  ISAF  forces  draw  down  and  transition  to  a  more  advisory  role— rather  than 
partnering  with  the  ANSF  to  jointly  engage  in  combat— the  United  States  and  its  coalition  partners  will  have 
fewer  of  their  own  advisors  conducting  assessments  and  will  have  to  rely  increasingly  on  the  ANSF  to  conduct 
its  own  assessments.  This  will  introduce  new  challenges  into  the  assessment  process,  including  more  limited 
ISAF  oversight  of  Afghan  capability  assessment  efforts.  With  the  security  transition  scheduled  to  be  completed 
by  the  end  of  2014,  there  is  a  narrow  window  of  opportunity  for  coalition  forces  to  implement  a  plan  that  will 
help  enable  the  continued  collection,  validation,  analysis,  and  reporting  on  the  status  of  the  ANSF’s  manning, 
training,  and  equipping  capability.  Without  such  a  plan,  decision  makers  in  the  United  States,  other  coalition 
countries,  and  the  Afghan  government  may  not  have  reasonable  assurance  that  the  ANSF  is  making  progress 
toward  becoming  a  self-sustaining  force  that  is  capable  of  assuming  full  security  responsibility  for  Afghanistan 
after  the  transition  is  completed  in  December  2014.  Further,  without  an  updated  plan  DOD  may  not  be  able  to 
make  fully  informed  decisions  based  on  ANSF  capability 

RECOMMENDATIONS 

To  ensure  that  assessments  of  the  ANSF  continue  during  and  after  the  drawdown  of  U.S.  and  coalition  military 
forces  in  Afghanistan,  we  recommend  that  the  Commander  of  the  International  Security  Assistance  Force: 

1.  Develop  and  implement  a  formal,  written  plan  for  collecting,  validating,  analyzing,  and  reporting  on 
ANSF  capabilities  that  details  how  ISAF  will 

a.  assess  the  ANSF’s  capability  with  fewer  advisors  to  conduct  the  assessments  and 

b.  mitigate  the  challenges  associated  the  ANSF  reporting  its  own  assessment  results. 

AGENCY  COMMENTS 

U.S.  Forces-Afghanistan  (USFOR-A)  provided  written  comments  on  a  draft  of  this  report,  which,  along  with  our 
responses,  are  reproduced  in  appendix  IV.  USFOR-A  concurred  with  the  "overall”  recommendations  of  our  audit 
report,  but  also  stated  that  it  did  not  concur  with  "some”  specific  recommendations.  In  addition,  IJC  provided 
us  informal  comments  on  a  draft  of  this  report,  which  we  have  incorporated,  as  appropriate. 

Our  draft  report  recommended  that  the  Commander  of  the  International  Security  Assistance  Force  develop  and 
issue  guidance— by  the  end  of  April  2014— that  provides  clear  instruction  on  the  level  of  detail  advisor  teams 
are  to  include  in  the  comments  supporting  their  assigned  rating  definition  levels  for  the  ANSF  units  they 
assess.  USFOR-A  did  not  concur  with  this  recommendation,  calling  it  "unnecessary.”  Specifically,  USFOR-A 
noted  that  IJC  issued  a  fragmentary  order  with  specific  instructions  on  the  information  the  advisor  units  are  to 
include  in  the  comments  supporting  their  assigned  rating  in  each  category  for  the  assessed  unit.  This 
fragmentary  order  directs  that  each  comment  should  focus  on  answering  three  questions,  the  answers  to 
which  "provide  not  only  the  what,  but  the  why  behind  the  issues  that  need  addressing  by  the  coalition  and 
confirm  the  accuracy  of  the  [Regional  Command’s]  assessment  of  a  particular  unit.”  USFOR-A  further  stated 
that  IJC  conducts  a  quality  control  check  of  every  narrative  to  ensure  adequate  information  is  provided.  Based 
on  USFOR-A’s  comment  and  the  referenced  fragmentary  order,  we  agree  that  the  recommended  guidance  has 
already  been  provided  to  the  advisor  units  and,  therefore,  we  have  deleted  this  recommendation  and  revised 
the  report  as  appropriate. 

Regarding  our  recommendation  to  develop  and  implement  a  formal,  written  plan  for  collecting,  validating, 
analyzing,  and  reporting  on  ANSF  capabilities,  USFOR-A  stated  that  "IJC  will  work  with  ISAF  to  codify  all  ongoing 
efforts  into  a  single  document  that  acts  as  an  overarching  roadmap  of  the  way  ISAF  intends  to  retain  the 
capability  of  assessing  the  progress  of  the  ANSF  now  and  in  the  future.”  It  appears  that  the  creation  of  such  a 
document  would  mostly  satisfy  our  recommendation.  We  will  follow  up  with  USFOR-A  to  review  this  document 
once  it  is  complete  and  determine  at  that  time  whether  our  recommendation  has  been  implemented. 
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APPENDIX  I  -  SCOPE  AND  METHODOLOGY 


The  objectives  of  this  audit  were  to  assess  the  extent  to  which  (1)  the  International  Security  Assistance  Force’s 
(ISAF)  tool  for  evaluating  the  manning,  training,  and  equipping  capacity  of  the  ANSF— the  Commander’s  Unit 
Assessment  Tool  (CUAT)— was  consistently  applied  to  assess  ANSF  units,  and  (2)  ISAF  is  taking  steps  to  plan  for 
the  continued  collection,  validation,  analysis,  and  reporting  of  ANSF  capacity  assessments  during  the  military 
drawdown  and  the  transition  of  security  responsibility  to  the  Afghan  government.  We  did  not  conduct  an 
independent  assessment  of  the  ANSF’s  capacity  to  man,  train  and  equip  its  forces.  Rather,  we  determined 
whether  the  tools  ISAF  used  to  assess  the  ANSF’s  capacity  to  man,  train,  and  equip  its  forces  provided  users 
with  sufficient  information  to  inform  decision  making. 

To  assess  the  extent  to  which  ISAF’s  tool  for  assessing  the  manning,  training,  and  equipping  was  consistently 
applied,  we  reviewed  the  National  Defense  Authorization  Act  for  Fiscal  Year  2008,  as  amended,  which  directs 
the  Department  of  Defense  (DOD)  to  report  on  the  capabilities  of  the  ANSF.29  We  also  reviewed  the  April  2010 
United  States  Plan  for  Sustaining  the  Afghanistan  National  Security  Forces,  the  NATO  security  force  assistance 
model,  the  DOD  Report  on  Progress  Toward  Security  and  Stability  in  Afghanistan,  and  the  Joint  Center  for 
International  Security  Force  Assistance  Afghan  National  Security  Forces  (ANSF)  Advisor  Guide.30  We  reviewed 
the  CUAT  Standard  Operating  Procedure,  fragmentary  orders,  and  current  and  past  rating  definition  levels  used 
to  assess  ANSF  capacities.  We  reviewed  CUAT  results  and  summary  data  from  April  2011  through  July  2013.  In 
addition,  we  analyzed  CUAT  rating  definition  level  titles  and  supporting  standards  from  April  2010  through  July 
2013  in  order  to  determine  the  extent  to  which  the  standards  changed  as  the  titles  changed.  To  do  this,  we 
listed  all  of  the  standards  for  each  rating  definition  level,  compared  them  against  each  other,  and  identified 
consistencies  and  inconsistencies.  Individual  assessments  are  classified,  and  as  a  result,  we  could  not  provide 
specific  information  contained  in  them.  We  visited  select  locations  in  Kandahar,  Flelmand,  and  Kabul  provinces 
in  Afghanistan  to  meet  with  coalition  advisor  teams;  observe  capacity  assessment  data  collection,  analysis, 
and  verification  processes;  and  review  capacity  assessment  documents,  which  are  classified.  We  reviewed 
preliminary  information,  such  as  fragmentary  orders  and  a  decision  brief,  on  the  Regional  ANSF  Status  Report, 
ISAF  Joint  Command’s  (IJC)  new  assessment  system  that  replaced  the  CUAT.  We  also  reviewed  a  prior  SIGAR 
audit  on  ANSF  assessments,31  and  reports  and  testimonies  produced  by  other  audit  agencies.32  We 
interviewed  officials  from  the  Office  of  the  Under  Secretary  of  Defense  for  Policy,  the  Office  of  the  Joint  Chiefs 
of  Staff,  U.S.  Central  Command,  the  IJC  Campaign  Transition  Assessment  Group,  and  the  ISAF  Afghan 
Assessments  Group.  We  also  visited  Security  Force  Assistance  Advisor  Teams  in  Regional  Commands  South, 
Southwest,  and  Capital  to  obtain  additional  perspectives  on  the  collection,  validation,  analysis,  and  reporting  of 
CUAT  assessments.33 

To  assess  the  extent  to  which  ISAF  is  taking  steps  to  plan  for  the  continued  collection,  validation,  analysis,  and 
reporting  of  ANSF  capacity  assessments  during  the  military  drawdown  and  transition  of  security  responsibility, 
we  reviewed  the  2010  United  States  Plan  for  Sustaining  the  Afghanistan  National  Security  Forces,  the  NATO 
security  force  assistance  model,  the  DOD’s  reports  to  Congress  on  the  progress  toward  security  and  stability  in 
Afghanistan,  and  the  November  2012  Afghanistan  Assessments  Conference  Report  and  task  list.  We  also 
reviewed  NATO’s  Combined  Joint  Statement  of  Requirements,  version  13.0,  which  outlines  NATO’s  required 


29  Pub.  L.  No.  110-181,  §§1230-1231,  122  Stat.  3,  385-391  (2008)  (amended  2013). 

30  Joint  Center  for  International  Security  Force  Assistance,  Afghan  National  Security  Forces  (ANSF)  Advisor  Guide  (Version  2),  April  2013, 

31  SIGAR  Audit  10-11,  Actions  Needed  to  Improve  the  Reliability  of  Afghan  Security  Force  Assessments,  June  29,  2010. 

32  U.S.  Government  Accountability  Office,  Afghanistan:  Key  Oversight  Issues,  GA0-13-218SP,  February  11,  2013;  DOD  Inspector  General, 
Assessment  of  the  U.S.  Government  to  Train ,  Equip ,  and  Mentor  the  Expanded  Afghan  National  Police,  SP0-2011-003,  March  3,  2011; 
U.S.  Government  Accountability  Office,  Statement  of  Charles  M.  Johnson  and  Sharon  L.  Pickup,  Afghanistan  Security:  Long-standing 
Challenges  May  Affect  Progress  and  Sustainment  of  Afghan  National  Security  Forces ,  GA0-12-951T,  July,  24,  2012;  DOD  Inspector 
General,  Statement  of  Ambassador  (Ret.)  Kenneth  P.  Moorefield,  Afghan  National  Security  Forces  and  Security  Lead  Transition:  The 
Assessment  Process ,  Metrics  and  Efforts  to  Build  Capacity,  July  24,  2012. 

33  We  attempted  to  visit  additional  advisor  teams  in  Regional  Commands  North  and  West,  but  were  restricted  due  to  weather  and  security. 
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staffing  levels  for  its  advising  mission,  among  other  things.  In  addition,  we  interviewed  officials  from  IJC,  the 
Office  of  the  Under  Secretary  of  Defense  for  Policy,  the  Office  of  the  Joint  Chiefs  of  Staff,  and  U.S.  Central 
Command. 

We  did  not  use  or  rely  on  computer-processed  data  for  the  purposes  of  the  audit  objectives.  With  respect  to 
assessing  internal  controls,  as  part  of  our  first  objective,  we  reviewed  the  CUAT  Standard  Operating  Procedure 
and  fragmentary  orders  to  determine  whether  they  provided  sufficient  to  instructions  to  advisor  teams 
completing  the  assessments.  The  results  of  our  review  are  included  in  the  body  of  this  report. 

We  conducted  our  audit  work  in  Kabul,  Helmand,  and  Kandahar  provinces  in  Afghanistan,  and  Washington, 
D.C.,  from  February  2013  to  February  2014,  in  accordance  with  generally  accepted  government  auditing 
standards.  Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain  sufficient,  appropriate 
evidence  to  provide  a  reasonable  basis  for  our  findings  and  conclusions  based  on  our  audit  objectives.  We 
believe  that  the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and  conclusions  based  on  our 
audit  objectives.  This  audit  was  performed  by  SIGAR  under  the  authority  of  Public  Law  No.  110-181,  as 
amended,  and  the  Inspector  General  Act  of  1978,  as  amended. 
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APPENDIX  II  -  CUAT  QUALITATIVE  ASSESSMENT  AREAS 


Under  the  Commander’s  Unit  Assessment  Tool  (CUAT),  International  Security  Assistance  Force  Joint  Command 
advisor  teams  assessed  the  Afghan  National  Security  Forces  according  to  one  overall  assessment  with  14 
component  assessment  areas.  Each  area  included  a  series  of  topics  for  advisors  to  comment  on  to  support  the 
assigned  the  CUAT  rating  definition  level.  Figure  2  lists  the  CUAT’s  qualitative  assessment  areas  and  topics  for 
the  advisors  to  comment  on. 


Figure  2  -  CUAT  Qualitative  Assessment  Areas  and  Topics 


OVERALL  ASS  ESS  ME  PVT. 

•  Months  to  next  RDL  Estimated  time  for  the  unit  to  progress 
to  the  next  RDL  (in  months),  based  on  the  assessor's  experi¬ 
ence.  This  is  not  required  if  the  unit  is  rated  as  Independent 
with  Advisors  (RDL  1 ) 

•  Must  include  comments  on  the  following: 

>  Time  to  progress  to  next  RDL  and  why, 

)  Key  factors  limiting  or  enabling  overall  assessment 
(operations,  intelligence,  logistics,  leadership,  etc.), 

>  Actions  needed  to  accelerate  the  development  of  the  unit 
)  And  an  explanation  for  change  in  previous  overall  assess¬ 
ment  rating  if  applicable 

OPERATION. 

Must  include  comments  on  the  following: 

•  Unit’s  mission, 

•  Ability  to  accomplish  the  mission 

•  Employment  of  basic  Operations  Security  (OPSEC) 

•  Ability  to  plan  or  conduct  basic  troop  leading  procedures 
and/or  military  decision  making  process  (MDMP) 

•  Ability  to  coordinate  with  external  units  and  agencies 

•  Ability  to  conduct  MEDEVAC/CASEVAC 

•  Ability  to  treat  casualties 

•  Ability  to  execute  Counter  IED  (CIED)  operations, 

•  And  ability  to  maneuver  within  its  area  of  responsibility. 

PARTNERING. 

Must  include  comments  on  the  following: 

•  Time  with  the  unit 

•  Frequency  of  visits 

•  Confidence  in  assessment 

•  Embedded  or  patrolling  to  advise 

•  Afghan  dependency  on  PU/AT 

•  Assessment  of  the  relationship  between  PU/AT  and  ANSF 
unit  leadership 

•  Any  planned  changes  to  relationship  in  the  next  cycle 

•  And  is  the  unit  temporarily  deployed 

LEADERSHIP. 

Must  include  comments  on  the  following: 

•  Are  leaders  present  and  do  they  actively  participate  in  plan¬ 
ning  and  execution  of  operations 

•  Do  they  enable  subordinate  leaders  to  accomplish 
themission 

•  Does  leadership  negatively  impact  mission  accomplishment? 

•  Leadership  competence 

•  Unit  morale  (including  health  and  welfare) 

•  AN  SF  will  to  fight 

•  Actions  taken  to  remove  or  correct  negative  leaders 


•  And  external  assistance  needed  for  removing  or  correcting- 
negative  leaders 

•  Competence,  Honesty,  Influence  and  Loyalty  RDLs.  Select 
RDL  from  drop  downs  based  on  the  definitions  in  Annex  D. 

TRAINING  AND  EDUCATION. 

Used  as  a  workaround  for  training  data  under  Host  Nation 
Resources,  pending  form  redesign. 

•  Format  of  numbers  within  #  symbols  are  required  so 
that  data  may  be  queried  using  WebTAS.  Will  include  at 
a  minimum:  personnel  requiring  branch  specific  train¬ 
ing  by  specialty  MOS  (#MOS  Auth/OHITrained#),  PME  or 
Institutional  Police  Education  completion  (by  Officer,  NCO. 
and  Enlisted:  #PME  Auth/OHITrained#) 

•  Additional  Comments: 

>  Workarounds  and  challenges  due  to  illiteracy 

>  Observed  training  best  practices 

)  Status  of  CIED  and  IED  defeat/awareness  training 
)  And  unit’s  plan  for  getting  untrained  personnel 
trained(especially  for  enabler  functions  and  specialty 
MOSs) 

>  Specialty  MOSs  required  include:  EN,  EOD.  Medical, 
Signal,  Ml,  Fires  (artillery  and  mortar),  Logistics 

MAINTENANCE,  AND  SUPPLY. 

Similar  police  specialties  are  also  required. 

CORRUPTION  AND  DRUG  USE. 

RDL.  Pending  update.  Observed  or  Not  Observed. 

Must  include  comments  on  the  following : 

•  Has  the  AT  or  PU  observed  anything  (like  selling  of  fuel  or 
equipment) 

•  And  if  so.  did  it  impact  the  unit’s  ability  to  accomplish  the 
mission 

COMMUNICATIONS 

Must  include  comments  on  the  following: 

•  Does  the  unit  have  reliable  internal/external  communications 

•  Ability  to  communicate  using  secure  means/methods 

•  Have  operations  been  compromised  due  to 
communicationmethods 

•  Status  of  MoD/Mol  nets  (KDK  and  PHQ  and  above) 

•  Effectiveness  and  limitations  of  communications 

•  And  ability  to  use  and  operate  authorized 
communicationsequipment 

PERSONNEL/ TRAINING. 

Not  required  and  pending  deletion. 

Not  Assessed  in  the  meantime. 
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EQUIPMENT  HOLDINGS/READINESS. 

•  Not  required  and  pending  deletion.  Select  Not  Assessed  in  the 
meantime. 

EQUIPPING. 

Used  as  a  workaround  for  materiel  data  underHost  Nation 

Resources. 

•  Format  of  numbers  within  #  symbols  are  required  so  that  data 
may  be  queried.  Will  include  at  a  minimum:  totals  for  shoot, 
move,  communicate  (#ETOT  Auth/OH/Operational#),  status 
of  mission  essential  equipment  by  type  (#EME  Auth/OH/ 
Operational#) 

•  Additional  Comments 

>  Accountability  of  equipment  (especially  sensitive  items 
andmission  essential  equipment) 

)  Fielding  and  status  of  CIEO  equipment  [metal  detectors 
andElectronic  Counter  Measures  (ECMs)] 

)  And  unit’s  plan  for  completing  equipment  fielding  (if 
needed) 

INTELLIGENCE. 

Must  include  comments  on  the  following: 

•  Does  the  unit  have  established  Priority  Intelligence 
Requirements  (PIR) 

•  Unit’s  ability  to  collect  and  dissemination  information  in  sup¬ 
port  of  PIR 

•  And  ability  to  use  information  collected  to  drive  operations 

LOGISTICS. 

Must  include  comments  on  the  following: 

•  Unit’s  overall  logistics  assessment 

•  Time  to  progress  to  next  logistics  RDL  and  why 

•  Key  factors  limiting  or  enabling  logistics 

•  Explanation  for  change  in  previous  overall  assessment  rating 
if  applicable 

•  And  ability  to  plan  logistics  (including  medical)  in  support  of 
operations 

MAINTENANCE. 

Must  include  comments  on  the  following: 

•  Unit’s  ability  to  perform  maintenance 


•  Status  of  contract  maintenance  (ANP) 

•  Training  of  ANP  to  perform  maintenance 

•  And  plans  for  training  maintained  (as  needed) 

PERSONNEL. 

Used  as  a  workaround  for  personnel  data  under  Host  Nation 
Resources. 

•  Format  of  numbers  within  #  symbols  are  required  so  that  data 
may  be  queried.  Will  include  at  a  minimum:  personnel  assign¬ 
ment  and  utilization  for  specialty  MOSs  (#UTIL  Auth/OH/Filling 
Auth  Position#) 

•  Additional  Comments: 

>  Ability  to  track  personnel  and  assignments 

>  Ability  to  pay  personnel  accurately  and  on  time 

>  Ability  to  incorporate  and  track  leave  and/or  the  red/ 
amber/green  cycle 

>  And  observed  challenges  and  best  practices 

TRANSPORTATION. 

Pending  change.  This  field  will  be  changed  to  Supply  in  CIDNE. 
However,  pending  that  change,  the  transportation  field  will 
beused  to  provide  Supply  RDLs  and  Comments. 

•  Must  include  comments  on  the  following: 

>  Ability  to  track  and  account  for  classes  of  supply 

>  Ability  to  use  the  Afghan  supply  system  (including  process¬ 
ing  of  requests  and  forms) 

>  Observed  weak  points  in  the  supply  system 

>  And  observations  on  supply  failures. 

INFRASTRUCTURE/FACILITIES. 

Must  include  comments  on  the  following: 

•  Does  the  unit  have  adequate  facilities  for  housing,  dining, 
medical,  maintenance,  and  operations? 

•  If  not,  what  are  the  major  limitations? 

•  What  is  being  done  to  fix  any  issues? 

•  Are  their  facilities  permanent  or  temporary? 

•  Is  their  infrastructure  sustainable,  and  why  or  why  not? 

•  And  are  there  local  facilities  that  can  and/or  do  fill  gaps  in 
capabilities? 


Source:  Commander’s  Unit  Assessment  Tool  Standard  Operating  Procedure  (SOP),  CIDNE  Version  2. 1.8. 0.3,  IJC  ANSF  Assessments, 
February  26,  2013. 
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APPENDIX  III  -  CUAT  RATING  DEFINITION  LEVELS 


Using  the  Commander’s  Unit  Assessment  Tool  (CUAT),  International  Security  Assistance  Force  Joint  Command 
advisor  teams  assessed  the  Afghan  National  Security  Forces  (ANSF)  according  one  overall  assessment  and  14 
component  assessment  areas.  CUAT  rating  definition  levels  are  the  overall  subjective  rating  of  these  15  areas 
based  on  the  advisor  teams’  assessment.  Figure  3  shows  descriptions  of  the  overall  CUAT  rating  definition 
levels  that  were  in  effect  from  August  2011  to  July  2013. 


Figure  3  -  Overall  CUAT  Rating  Definition  Levels  Starting  in  August  2011 

Independent  with  Advisors.  The  unit  Is  able  to  plan  and  execute  Its  missions  and  maintain 
command  and  control  of  subordinate  elements,  calling  on  and  controlling  a  QRF  and  MEDEVAC 
assets  when  required.  When  necessary,  the  unit  can  call  for  and  integrate  joint  effects  from 
coalition  forces  and  is  capable  of  exploiting  intelligence  and  operating  within  a  wider  intelligence 
system.  Unit  personnel  and  equipment  levels  arei  75%  of  the  authorized  strength  (Present  for  Duty/ 
Operational)  and  are  sufficient  for  the  unit  to  undertake  its  assigned  tasks.  The  unit  can  operate,  account 
Tor  and  maintain  its  equipment  and  meet  its  basic  logistic  needs  with  no  assistance  from  Coalition  Forces. 
The  unit  is  able  to  co-ordinate  its  operations  with  other  ANSF  units  and  headquarters. 

~~  Effective  with  Advisors.  The  unit  conducts  effective  planning,  synchronizing,  directing,  and 
reporting  of  operations  and  status.  The  unit  is  able  to  coordinate  and  communicate  with  higher, 

I  lower,  adjacent,  and  combined/joint  units.  The  unit  maintains  effective  readiness  reports. 
Leaders,  staff,  and  unit  adhere  to  the  ANSF  Code  of  Conduct,  and  are  loyal  to  GIROA.  All  unit 
enablers  are  present  and  effective  in  this  unit,  and  are  providing  most  of  the  support  to  the  ANSF  unit. 
Coalition  Forces  provide  only  limited,  occasional  guidance  to  the  staff.  Coalition  Forces  may  provide 
enablers  that  are  missing  from  HIGHER  or  LOWER  ANSF  levels.  Coalition  Forces  augment  support  only  on 
occasion.  Equipment  and  personnel  levels  are  >:75%.  OCC  P  and  R  are  always  integrated,  and  links  are 
usually  effective. 

Effective  with  Partners.  The  unit  requires  routine  mentoring  for  planning,  synchronizing, 
directing,  and  reporting  of  operations  and  status:  coordinating  and  communicating  with  higher, 
lower,  adjacent,  and  comblned/Joinl  units:  and  maintaining  effective  readiness  reports.  Leaders, 
staff,  and  most  of  the  unit  adhere  to  the  ANSF  Code  of  Conduct,  and  aie  loyal  to  GIROA.  Most 
unit  enablers  are  present  and  effective.  Those  enablers  present  are  providing  most  of  the  support  to  the 
ANSF  unit.  Coalition  Forces  may  provide  enablers  that  are  missing  from  this  level,  and  augment  enabler 
support  at  this  level.  Equipment  and  personnel  levels  are  >65%.  OCC  P  and  R  are  always  integrated,  and 
links  are  sometimes  effective. 

■  Developing  with  Partners.  The  unit  requires  partner  unit  presence  and  assistance  for 

planning,  synchronizing,  directing,  and  reporting  of  operations  and  status;  coordinating  and 
communicating  with  higher,  lower,  adjacent,  and  coinbined/joint  units;  and  maintaining  effective 
readiness  reports.  Leaders  and  most  of  the  staff  usually  adhere  to  the  ANSF  Code  of  Conduct, 
and  are  loyal  to  GIROA.  Some  unit  enablers  are  present  and  effective  at  this  level  of  unit.  Those  present 
provide  some  of  the  support  to  the  ANSF  unit.  Coalition  Forces  provide  the  missing  enablers  and  most  of 
the  support  at  this  level.  Equipment  and  personnel  levels  are  <65%.  OCC  P  and  R  are  usually  integrated. 

Established.  This  unit  is  at  the  beginning  of  organization  or  has  just  been  fielded.  It  is  barely 
capable  of  planning,  synchronizing,  directing,  or  reporting  operations  and  status,  even  with  the 
presence  and  assistance  of  a  partner  unit.  It  is  barely  able  to  coordinate  and  communicate  with 
higher,  lower,  adjacent,  and  cornbmed/joint  units.  It  has  barely  effective  or  missing  personnel, 
logistics,  training,  or  readiness  reports.  Leadership  and  staff  may  not  adhere  to  the  AN  SF  Code  of 
Conduct,  or  may  not  be  loyal  to  GIROA.  Most  of  the  units'  unit  enablers  are  not  present  or  are  barely 
effective.  Those  enablers  present  provide  little  or  no  support  to  the  ANSF  unit.  Coalition  Forces  provide  for 
the  missing  enablers  and  most  of  the  support  at  this  level.  Equipment  and  personnel  levels  are  <50%.  OCC 
P  and  R  are  sometimes  integrated. 


Not  Assessed.  Areas  where  insufficient  data  is  available  for  complete  assessment 


Notes:  QRF=quick  reaction  force;  MEDEVAC= medical  evacuation;  GIROA=Government  of  the  Islamic  Republic  of 
Afghanistan;  OCC  P=Operational  Coordination  Center  Provincial;  and  OCC  R=Operational  Coordination  Center 
Regional. 

Source:  Commander’s  Unit  Assessment  Tool  Standard  Operating  Procedure  (SOP),  CIDNE  Version  2. 1.8. 0.3,  IJC  ANSF 
Assessments,  February  26,  2013. 
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APPENDIX  IV-  COMMENTS  FROM  U.S.  FORCES-AFGHANISTAN 


Mw 


REPLY  TO 

ATTENTION  OF: 

USFORA-DCG 


HEADQUARTERS 

United  States  Forces  -  Afghanistan  (USFOR-A) 
Kabul,  Afghanistan 
APO  AE  09320 


24  January  2014 


MEMORANDUM  FOR:  SPECIAL  INSPECTOR  GENERAL  for  AFGHANISTAN 
RECONSTRUCTION  (SIGAR),  Crystal  City,  Virginia  22202 

SUBJECT:  USFOR-A  Concurrence  with  Comments  to  SIGAR  Draft  Audit  Report  14-x.  ’‘Afghan 
National  Security  Forces:  Actions  Needed  to  Improve  Capability  Assessments  and  Plans  for  Sustaining 
Assessment  Efforts”,  dated  Januarv  2014 


1 .  USFOR-A  acknowledges  and  appreciates  SIGAR's  recognition  of  the  difficulties  in  assessing  the 
Afghan  National  Security  Forces  (ANSF)  as  Coalition  Forces  draw  down.  This  is  a  challenge  that  both 
ISAF  and  USFOR-A  are  addressing  through  a  scries  of  efforts  directed  toward  both  the  ANSF  and 
Coalition  Forces,  as  alluded  to  w  ithin  the  body  of  the  SIGAR  Report.  USFOR-A  concurs  with  the  overall 
recommendations  of  the  SIGAR  14-x  Audit  Report,  however  non-concurs  with  some  specific 
recommendations  and  provides  the  comments  below  for  further  consideration  and  clarification. 

2.  SIGAR  Recommendations: 

1.  Develop  and  issue  guidance  -  by  the  end  of  April  2014  -  that  provides  clear  instruction  on 
the  level  of  detail  advisor  teams  are  to  include  in  the  comments  supporting  their  assigned 
rating  definition  levels  for  the  ANSF  units  they  assess. 

USFOR-A  non-concurs  with  this  specific  recommendation  as  unnecessary.  IJC  FRAGO  244, 
issued  on  30  September  2013,  contains  specific  instructions  on  the  information  the  advisor  units 
arc  to  include  in  the  comments  supporting  their  assigned  rating  in  each  category  for  the  assessed 
unit. 

The  FRAGO  (mentioned  previously  by  COMIJC  in  his  letter  to  the  SIGAR  on  19  October  2013) 
directs  each  comment  to  focus  on  answering  three  questions.  The  answers  to  these  questions 
provide  not  only  the  what,  but  the  why  behind  the  issues  that  need  addressing  by  the  coalition  and 
confirm  the  accuracy  of  the  RC’s  assessment  of  the  particular  unit.  The  answers  to  these  directed 
questions  also  provide  COMIJC  and  COM  ISAF  the  necessary  information  to  allocate  scarce 
resources  and  make  necessary  decisions.  Additionally,  IJC  conducts  a  quality  control  check  on 
every  narrative  to  ensure  adequate  information  is  provided. 

2.  Develop  and  implement  a  formal,  w  ritten  plan  for  collecting,  validating,  analyzing,  and 
reporting  on  ANSF  capabilities  that  details  how  ISAF  will: 

a.  Assess  the  ANSF’s  capability  with  fewer  advisors  to  conduct  the  assessments  and 

b.  Mitigate  the  challenges  associated  with  the  ANSF  reporting  its  own  assessment  results. 

IJC  will  work  with  ISAF  to  codify  all  of  the  ongoing  efforts  into  a  single  document  that  acts  as  an 
overarching  road  map  of  the  way  ISAF  intends  to  retain  the  capability  of  assessing  the  progress  of 
the  ANSF  now  and  into  the  future. 


NATO/ISAF  UNCLASSIFIED 


See  SIGAR 
comment  1. 


See  SIGAR 
comment  2. 


See  SIGAR 
comment  3. 
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SUBJECT:  Non-concurrence  with  the  recommendations  of  the  SIGAR  14-x  Audit  Report 


3.  SIGAR's  report  is  focused  on  the  Commander's  Unit  Assessment  Tool  (CUAT).  which  is  a  report  that 
has  not  been  in  use  since  June  2013.  While  the  Regional  ANSF  Status  Report  (RASR),  and  the  CUAT 
before  that,  provides  a  common  and  recurring  foundation  for  making  assessments  of  the  ANSF,  the  final 
assessment  is  a  combination  of  a  number  of  reports,  both  qualitative  and  quantitative,  to  include 
commander's  updates.  Any  external  review  of  ISAF's  and  IJC’s  ability  to  assess  the  ANSF  must  take 
these  other  mechanisms  into  account  in  order  to  provide  informative  and  relevant  feedback. 


See  SIGAR 
comment  4. 


4,  Point  of  Contact  for  this  memo  is  COL  (OF-5)  Ronald  McNamara,  UC  Inspector  General,  DSN: 

3rCOL(OF-5)  Wesley  Jennings,  Director, 


email: 

1JC  Campaign  and  Transition  Assessment  Group,  DSN:  I 


[  email: 


MARK  A.  MILLEY 
Lieutenant  General,  USA  \ 

Deputy  Commander,  United  States  Forces  -  Afghanistan 


NATO/I SAF  UNCLASSIFIED 
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SIGAR’s  Response  to  Comments  from  USFOR-A 

1.  See  comment  2. 

2.  Based  on  USFOR-A’s  comment  and  our  review  of  the  referenced  fragmentary  order,  we  agree  that  the 
recommended  guidance  has  been  provided  to  the  advisor  teams.  We  have  deleted  the  recommendation 
and  revised  the  report  as  appropriate.  For  example,  we  revised  the  title  to  reflect  the  deletion  of  the 
recommendation. 

3.  The  document  USFOR-A  describes  in  its  comments  would  largely  satisfy  our  recommendation.  We  will 
follow  up  with  USFOR-A  to  review  this  document  once  it  has  been  completed  and  determine  at  that  time  if 
our  recommendation  has  been  implemented. 

4.  As  noted  in  our  report,  our  first  objective  was  specifically  to  assess  the  extent  to  which  ISAF’s  tool  for 
evaluating  the  manning,  training,  and  equipping  capacity  of  the  ANSF— the  CUAT— was  consistently  applied 
to  assess  ANSF  units.  We  were  not  informed  officially  until  late  August  2013  that  IJC  had  replaced  the 
CUAT  with  the  RASR.  Despite  having  already  completed  our  fieldwork,  we  revised  our  draft  report  to  include 
information  on  the  RASR  in  an  effort  to  recognize  the  steps  IJC  had  taken  to  improve  its  assessment 
efforts,  particularly  in  light  of  the  drawdown  of  coalition  forces.  Although  IJC  stopped  using  the  CUAT  in 
June  2013,  IJC’s  response  to  SIGAR’s  prior  audit  of  ANSF  capability  assessments  suggests  that  our 
findings  are  still  relevant  to  the  improvement  of  the  assessment  process.34  In  particular,  shortly  before  the 
completion  of  our  2010  audit  of  IJC’s  assessments  of  the  ANSF,  the  command  introduced  a  new  unit 
assessment  tool— the  CUAT— that  replaced  the  CM  tool.  Flowever,  that  particular  change  in  assessment 
tools  did  not  negate  the  importance  of  our  findings  on  the  CM  because,  in  responding  to  that  report,  IJC 
stated  it  would  address  our  findings  and  recommendations  as  part  of  its  implementation  of  the  CUAT. 
Similarly,  the  findings  in  our  current  report  are  relevant  to  the  RASR  and  any  future  ANSF  assessments 
tools  because  they  inform  IJC  of  weaknesses  with  the  CUAT.  This  information  could  enable  the  command 
to  take  actions  to  prevent  these  weaknesses  from  resurfacing.  Finally,  we  do  not  suggest  that  the  CUAT 
and  its  successor  the  RASR  are  the  only  sources  of  information  used  to  assess  the  ANSF’s  capabilities.  Our 
draft  report  clearly  noted  that  IJC  used  the  results  of  the  assessments  along  with  other  information,  such 
as  battlefield  reporting,  informal  commander’s  assessments,  and  ISAF’s  assessments  of  the  Ministries  of 
Defense  and  Interior,  to  develop  an  overall  assessment  of  the  ANSF’s  operational  effectiveness  and 
readiness.  Although  the  RASR,  and  the  CUAT  before  it,  is  not  the  only  information  that  feeds  into  IJC’s 
overall  assessment  of  the  ANSF,  it  is  a  tool  for  regularly  collecting  a  standardized  set  of  information  on  the 
capability  of  ANSF  units.  As  such,  an  evaluation  of  the  tool  still  has  merit  in  providing  information  on  ISAF’s 
and  IJC’s  assessment  efforts. 


34  SIGAR  Audit  10-11,  Actions  Needed  to  Improve  the  Reliability  of  Afghan  Security  Force  Assessments,  June  29,  2010. 
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SIGAR’s  Mission 


The  mission  of  the  Special  Inspector  General  for  Afghanistan 
Reconstruction  (SIGAR)  is  to  enhance  oversight  of  programs  for  the 
reconstruction  of  Afghanistan  by  conducting  independent  and 
objective  audits,  inspections,  and  investigations  on  the  use  of 
taxpayer  dollars  and  related  funds.  SIGAR  works  to  provide  accurate 
and  balanced  information,  evaluations,  analysis,  and 
recommendations  to  help  the  U.S.  Congress,  U.S.  agencies,  and 
other  decision-makers  to  make  informed  oversight,  policy,  and 
funding  decisions  to: 

•  improve  effectiveness  of  the  overall  reconstruction 
strategy  and  its  component  programs; 

•  improve  management  and  accountability  over  funds 
administered  by  U.S.  and  Afghan  agencies  and  their 
contractors; 

•  improve  contracting  and  contract  management 
processes; 

•  prevent  fraud,  waste,  and  abuse;  and 

•  advance  U.S.  interests  in  reconstructing  Afghanistan. 


Obtaining  Copies  of  SIGAR 
Reports  and  Testimonies 


To  obtain  copies  of  SIGAR  documents  at  no  cost,  go  to  SIGAR’s  Web 
site  (www.sigar.mil).  SIGAR  posts  all  publically  released  reports, 
testimonies,  and  correspondence  on  its  Web  site. 


To  Report  Fraud,  Waste,  and 
Abuse  in  Afghanistan 
Reconstruction  Programs 


To  help  prevent  fraud,  waste,  and  abuse  by  reporting  allegations  of 
fraud,  waste,  abuse,  mismanagement,  and  reprisal,  contact  SIGAR’s 
hotline: 

•  Web:  www.sigar.mil/fraud 

•  Email:  sigar.pentagon.inv.mbx.hotline@mail.mil 

•  Phone  Afghanistan:  +93  (0)  700-10-7300 

•  Phone  DSN  Afghanistan:  318-237-3912  ext.  7303 

•  Phone  International:  +1-866-329-8893 

•  Phone  DSN  International:  312-664-0378 

•  U.S.  fax:  +1-703-601-4065 


Public  Affairs 


Public  Affairs  Officer 

•  Phone:  703-545-5974 

•  Email:  sigar.pentagon.ccr.mbx.public-affairs@mail.mil 

•  Mail:  SIGAR  Public  Affairs 
2530  Crystal  Drive 
Arlington,  VA  22202 


